Meta-analysis Recap

Meta-analysis Data

How should we pool effect sizes?

To get our meta-analysis result, we need to take a weighted average of effect sizes…

Meta-analysis Models

• Fixed effects and random effects models make different assumptions about how data were generated
• Model should approximate the underlying processes

Fixed Effects Assumption

• One true effect size (\(\theta_{\ F }\))
• Deviation from \(\theta_{\ F}\) is due to sampling error (\(\epsilon_{\ k}\))

Inverse-Variance Weighting

• Error (\(\epsilon_{\ k }\)) is inversely proportional to sample size \(n_{k}\).
• Estimates (\(\widehat{\theta}_{k}\)) with small \(\epsilon_{\ k }\) should be closer to true \(\theta\)
• These \(\widehat{\theta}_{k}\) should be given more weight for estimating \(\theta\)

Inverse-Variance Weighting

Study weight \(w_{k}\) is inversely related to the variance (\(SE^2_{k}\))

\[ w_{k}=\frac{1}{(SE)^{2}_{k}} \]

The pooled estimate \(\widehat{\theta}\) is the weighted average:

\[ \widehat{\theta} = \frac{\Sigma^K_{k=1}\widehat{\theta}_{k} w_{k}}{\Sigma^K_{k=1}w_{k}} \]

FEM Standard Error

To compute the standard error on our averaged result \(\widehat\theta\)

\[ SE(\widehat{\theta}) = \frac{1}{\Sigma^K_{k=1}w_{k}} \]

FEM Assumptions & Interpretation

Random Effects Assumption

• A universe of possible studies
• Each study has own true effect size (\(\theta_k\))
• \(\theta_{k}\) are randomly drawn from a distribution with mean true effect size \(\mu\) and variance (\(\tau^2\))

Random Effects Model

The true effect size of study \(k\) (\(\theta_{k}\)) is drawn from a distribution with mean \(\mu\) with error \(\zeta_{k}\).

\[ {\theta}_{k}=\mu+\zeta_{k} \]

The reported result (\(\widehat{\theta}_{k}\)) of a study is generated by the true effect size (\(\theta_{k}\)) plus sampling error (\(\epsilon_{k}\)).

\[ \widehat{\theta}_{k}=\theta_{k} +\epsilon_{k} \]

Random Effects Model

• We can conceptualize the REM having a hierarchical structure

REM weighting

• uses the between-study heterogeneity of true effects, \(\tau^2\)
• \(\tau^2\) increases the relative weight given to smaller studies

\[ w^{*}_{k}=\frac{1}{SE^{2}_{k} + \tau^2} \]

Estimating \(\tau^2\)

• Beyond the scope of this talk!
• Many procedures exist (16+ different estimators)
• DerSimonian and Laird method treated as default, controversially

Exchangeability Assumption

• There is no way to know what \(\zeta_{k}\) is a priori
• This assumption may be violated by hidden information, e.g., publication bias

Should we just always use the REM?

• For most scenarios in biology, medicine, psych., … use the REM.
• Only choose the FEM if you have clear reason to (rare)

A Test for Between-Study Heterogeneity

• Does the confidence interval for \(\tau^2\) include zero or not?
• If not, using REM is justified

References & Further Reading

• Cochrane Handbook for Systematic Reviews of Interventions

• Doing Meta-Analysis in R: A Hands-On Guide

• Methods to estimate the between‐study variance and its uncertainty in meta‐analysis

• Why I don’t use the term “fixed and random effects”

• Stats, Stat!: Meta-analysis

• An Introduction to Systematic Review and Meta-analysis: Everything you need to know in 59 minutes

Questions?